PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain

Biomedical language understanding benchmarks are the driving forces for artificial intelligence applications with large language model (LLM) back-ends. However, most current benchmarks: (a) are limited to English which makes it challenging to replicate many of the successes in English for other languages, or (b) focus on knowledge probing of LLMs and neglect to evaluate how LLMs apply these knowledge to perform on a wide range of bio-medical tasks, or (c) have become a publicly available corpus and are leaked to LLMs during pre-training. To facilitate the research in medical LLMs, we re-build the Chinese Biomedical Language Understanding Evaluation (CBLUE) benchmark into a large scale prompt-tuning benchmark, PromptCBLUE. Our benchmark is a suitable test-bed and an online platform for evaluating Chinese LLMs' multi-task capabilities on a wide range bio-medical tasks including medical entity recognition, medical text classification, medical natural language inference, medical dialogue understanding and medical content/dialogue generation. To establish evaluation on these tasks, we have experimented and report the results with the current 9 Chinese LLMs fine-tuned with differtent fine-tuning techniques

Contrastive Triple Extraction with Generative Transformer

Triple extraction is an essential task in information extraction for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end triple extraction task for sequence generation. Since generative triple extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on three datasets (i.e., NYT, WebNLG, and MIE) show that our approach achieves better performance than that of baselines.Comment: Accepted by AAAI 202

From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer

Knowledge graph completion aims to address the problem of extending a KG with missing triples. In this paper, we provide an approach GenKGC, which converts knowledge graph completion to sequence-to-sequence generation task with the pre-trained language model. We further introduce relation-guided demonstration and entity-aware hierarchical decoding for better representation learning and fast inference. Experimental results on three datasets show that our approach can obtain better or comparable performance than baselines and achieve faster inference speed compared with previous methods with pre-trained language models. We also release a new large-scale Chinese knowledge graph dataset AliopenKG500 for research purpose. Code and datasets are available in https://github.com/zjunlp/PromptKG/tree/main/GenKGC.Comment: Accepted by WWW 2022 Poste

Normal vs. Adversarial: Salience-based Analysis of Adversarial Samples for Relation Extraction

Recent neural-based relation extraction approaches, though achieving promising improvement on benchmark datasets, have reported their vulnerability towards adversarial attacks. Thus far, efforts mostly focused on generating adversarial samples or defending adversarial attacks, but little is known about the difference between normal and adversarial samples. In this work, we take the first step to leverage the salience-based method to analyze those adversarial samples. We observe that salience tokens have a direct correlation with adversarial perturbations. We further find the adversarial perturbations are either those tokens not existing in the training set or superficial cues associated with relation labels. To some extent, our approach unveils the characters against adversarial samples. We release an open-source testbed, "DiagnoseAdv" in https://github.com/zjunlp/DiagnoseAdv.Comment: IJCKG 202

LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training

Natural language generation from structured data mainly focuses on surface-level descriptions, suffering from uncontrollable content selection and low fidelity. Previous works leverage logical forms to facilitate logical knowledge-conditioned text generation. Though achieving remarkable progress, they are data-hungry, which makes the adoption for real-world applications challenging with limited data. To this end, this paper proposes a unified framework for logical knowledge-conditioned text generation in the few-shot setting. With only a few seeds logical forms (e.g., 20/100 shot), our approach leverages self-training and samples pseudo logical forms based on content and structure consistency. Experimental results demonstrate that our approach can obtain better few-shot performance than baselines.Comment: Work in progres

Construction and Applications of Billion-Scale Pre-trained Multimodal Business Knowledge Graph

Business Knowledge Graphs (KGs) are important to many enterprises today, providing factual knowledge and structured data that steer many products and make them more intelligent. Despite their promising benefits, building business KG necessitates solving prohibitive issues of deficient structure and multiple modalities. In this paper, we advance the understanding of the practical challenges related to building KG in non-trivial real-world systems. We introduce the process of building an open business knowledge graph (OpenBG) derived from a well-known enterprise, Alibaba Group. Specifically, we define a core ontology to cover various abstract products and consumption demands, with fine-grained taxonomy and multimodal facts in deployed applications. OpenBG is an open business KG of unprecedented scale: 2.6 billion triples with more than 88 million entities covering over 1 million core classes/concepts and 2,681 types of relations. We release all the open resources (OpenBG benchmarks) derived from it for the community and report experimental results of KG-centric tasks. We also run up an online competition based on OpenBG benchmarks, and has attracted thousands of teams. We further pre-train OpenBG and apply it to many KG- enhanced downstream tasks in business scenarios, demonstrating the effectiveness of billion-scale multimodal knowledge for e-commerce. All the resources with codes have been released at \url{https://github.com/OpenBGBenchmark/OpenBG}.Comment: OpenBG. Work in Progres

Laboratory and Experimental Hut Evaluation of a Long-lasting Insecticide Treated blanket for Protection against Mosquitoes.

Long-lasting insecticide treated blankets (LLIBs) may provide additional protection against malaria where use of long lasting insecticidal nets (LLIN) is low or impractical such as in disaster or emergency situations. Initial efficacy testing of a new candidate LLIB was carried out at LSHTM and KCMUCo, before and after washing, in cone and ball bioassays and arm-in-cage tests against pyrethroid susceptible Anopheles gambiae. A small scale field trial was conducted using veranda-trap experimental huts in northern Tanzania against wild An. arabiensis and Culex quinquefasciatus mosquitoes. Treatments included unwashed and 5 times washed permethrin treated LLIB and blankets hand-treated with permethrin (ITB), untreated blankets, and a holed unwashed Olyset net. Cone test mortality was 75% for LLIB when unwashed, but decreased to 32% after 5 washes and <10% after 10 washes. In arm-in-cage tests protection against biting was 100% for LLIBs regardless of the number of washes while reduction in landings was 79% when unwashed, 75% after 5 washes, but declined to 41% after 10 and 33% after 20 washes. In ball bioassays using pyrethroid resistant An. arabiensis, mortality was low in all treatments (<35%) and there was no significant difference in mortality between Olyset net, LLIB or ITB (p > 0.05). Percentage mortality of An. arabiensis in huts with LLIB unwashed (26%) was not statistically different to Olyset net (31%, p = 0.5). The 5 times washed LLIB reduced blood-feeding by 49% which was equivalent to Olyset net (p > 0.086). There was no significant difference in percentage blood-feeding between LLIB and ITB unwashed or 5 times washed (p = 0.147 and p = 0.346 respectively). The 5 times washed LLIB reduced blood-feeding of Culex quinquefasciatus by 40%, although the Olyset provided the greatest protection with 85% inhibition. ELISA analysis of a sub-sample of blood fed mosquitoes showed that not all had fed on humans in the huts, therefore blood-feeding inhibition may have been underestimated. This trial demonstrated the potential of LLIBs to provide substantial personal protection even against pyrethroid resistant mosquitoes. LLIBs may prove particularly useful where LLINs are unsuitable or net usage is low

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark

Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice. With the development of biomedical language understanding benchmarks, AI applications are widely used in the medical field. However, most benchmarks are limited to English, which makes it challenging to replicate many of the successes in English for other languages. To facilitate research in this direction, we collect real-world biomedical data and present the first Chinese Biomedical Language Understanding Evaluation (CBLUE) benchmark: a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification, and an associated online platform for model evaluation, comparison, and analysis. To establish evaluation on these tasks, we report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling. Our benchmark is released at \url{https://tianchi.aliyun.com/dataset/dataDetail?dataId=95414&lang=en-us}

Perceived dental treatment need among older Tanzanian adults – a cross-sectional study

Need perceptions for dental care play a key role as to whether people in general will seek dental care. The aim was to assess the prevalence of perceived need of problem based dental care, dental check-ups and any type of dental care. Guided by the conceptual model of Wilson and Cleary, the relationship of perceived need for dental care with socio-demographic characteristics, clinically defined dental problems and self-reported oral health outcomes was investigated. Partial prosthetic treatment need was estimated using a socio-dental approach. A cross-sectional survey was conducted in Pwani region and in Dar es Salaam in 2004/2005. Information from interviews and clinical examination became available for 511 urban and 520 rural adults (mean age 62.9 yr). 51.7% (95% CI 46.2, 57.0) urban and 62.5 % (95% CI 53.1, 70.9) rural inhabitants confirmed need for dental check-up, 42.9% (95% CI 36.9, 48.9) urban and 52.7% (95% CI 44.5, 60.6) rural subjects confirmed need for problem oriented care and 38.4% (95% CI 32.4, 44.6) urban versus 49.6% (95% CI 41.8, 57.4) rural residents reported need for any type of dental care. Binary and ordinal multiple logistic regression analyses revealed that adults who reported bad oral health and broken teeth were more likely to perceive need for dental care across the three outcome measures than their counterparts. Socio-demographic factors and clinically defined problems had less impact. Based on a normative and an integrated socio-dental approach respectively 39.5% and 4.7% were in need for partial dentures. About half of the participants confirmed need for problem oriented care, dental check-ups and any type of dental care. Need perceptions were influenced by perceived oral health, clinically assessed oral problems and socio-demographic characteristics. Need estimates for partial denture was higher when based on clinical examination alone compared to an integrative socio-dental approach